On Affect and Self-adaptation: Potential Benefits of Valence-Controlled Action-Selection

Psychological studies have shown that emotion and affect influence learning. We employ these findings in a machine-learning metaparameter context, and dynamically couple an adaptive agent’s artificial affect to its action-selection mechanism (Boltzmann β). The agent’s performance on two important learning problems is measured. The first consists of learning to cope with two alternating goals. The second consists of learning to prefer a later larger reward (global optimum) for an earlier smaller one (local optimum). Results show that, compared to several control conditions, coupling positive affect to exploitation and negative affect to exploration has several important benefits. In the alternatinggoal task, it significantly reduces the agent’s goal-switch search peak. The agent finds its new goal faster. In the second task, artificial affect facilitates convergence to a global instead of a local optimum, while permitting to exploit that local optimum. We conclude that affect-controlled action-selection has adaptation benefits.